The Trouble with Nonprofits
In the 1990s, a group of psychologists began studying what made experts expert. Their first task was to see whether experts really were expert — whether they were particularly good at their jobs.
What they found was that some were and some weren’t. Champion chess players, obviously, are much better at playing chess than you and I. But political pundits, it turns out, aren’t that much better at making predictions than a random guy off the street.
What distinguishes people who are great at what they do from those who are just mediocre? The answer, it seems, is feedback. If you lose a chess game, it’s pretty obvious you lost. You know right away, you feel bad, and you start thinking about what you did wrong and how you can improve.
Making a bad prediction isn’t like that. First, it’s months or years before your prediction isproven wrong. And then, you make yourself feel better by coming up with some explanation for why you were wrong:well, nobody expected that to happen; it threw everything else off!And so you keep on making predictions in the same way — which means you never get good at it.
The difference between chess and predictions is a lot like the difference between companies and nonprofits. If your company is losing money, it’s pretty obvious. You know right away, you feel bad, and you start thinking about how to fix it. (And if you don’t fix it, you go bankrupt.) But if your nonprofit isn’t accomplishing its goals, it’s much less obvious. You can point to various measurable signs of success (look at all the members we have, look at all the articles we’ve been quoted in) and come up with all sorts of explanations for why it’s not your fault.
This isn’t to say that we should have companies replace nonprofits, any more than we should have chess games replace predictions. The two serve completely different goals — nonprofits aim at improving the world, not making money. But it does mean that if you’re involved in nonprofits (or predictions), you need to be much more careful about making sure you’re doing a good job.
Unfortunately, few nonprofits do that. Take, for example, the Center for American Progress, widely believed to be one of the most effective political nonprofits. They say their goal is “improving the lives of Americans through ideas and action.” But their “marketing brochure,” while filled with glossy photos, doesn’t even attempt to see whether they’re accomplishing this goal. It touts that they’ve released “an economic strategy for the next administration,” “convened a task force … to develop policy,” and “developed a plan for the bulk transfer auction of at-risk mortgages.” There’s not a single attempt to demonstrate that any of these things has approved the lives of Americans, let alone estimate how much.
Measuring things is hard and expensive, even in the simplest cases. Measuring the effect of loaning money to Africans seems a lot easier than measuring the impact of of a think tank report. But when Peter Singer asked Oxfam to measure the effectiveness of giving microcredit to villages in West Africa, they declined, on the grounds that it would have taken up half the budget.
But not measuring is even more expensive. Imagine that Oxfam experimented with two microcredit programs and found that one did 10% better than the other. Even with this very modest improvement, it would only take helping five villages before the experiment paid for itself.
And, as anyone who’s done these sorts of experiments knows, you often see improvements well in excess of 10%. To take a silly example,Dustin Curtis experimentedwith getting more readers of his weblog to follow him on Twitter. After four experiments, he’d achieved a 173% improvement. And even this is probably underestimating things. I expect many nonprofits are not accomplishing their goalsat all. Even if they made a little bit of progress, their improvement would be mathematically infinite. (It’s also quite possible that many nonprofits are actually beingcounter-productive. After all, before we started measuring the effects of medical treatment, we were bleeding people with leeches.)
What can be done about this? I think that everyone who donates to a nonprofit should demand an accounting of results — not just the number of times they’ve been cited in the media or the number of policy discussions they’ve held, but an actual attempt to measure how much they’re improving people’s lives. For most nonprofits, I expect these numbers will be depressingly small. But that’s much better than having no numbers at all. For feeling bad about failing is the first step to doing better next time.
You should follow me on twitterhere.
September 7, 2009